Study 1: Measuring meritocracy with international secondary data

Data

The data used in the analysis corresponds to the last available wave of the social inequality module from the International Social Survey Programme (ISSP), which is the most specialized international comparative survey in perceptions, attitudes and beliefs about inequality related issues. This wave corresponds to the year 2009 and covers attitudes towards a series of topics dealing with social inequality across 41 countries and 56021 individuals. Although there is data available from this module for previous waves (1987, 1992 and 1999), unfortunately there are several variables that are important for this research that were not included, particularly in 1987 and 1999, reason why we only use the 2009 wave.

Variables

There are a series of indicators in the ISSP survey that in the following we attempt to classify in our meritocracy measurement scheme. However, it is important to mention that the items were not originally thought as a specific measure of meritocracy. Still, many of the items have been used in previous research on meritocracy (and certainly will keep being used in the future) and before proposing a new measure we consider to give a first test to our proposed conceptualization using available data that not only covers several countries but also is validated by the research community.

The variables are presented below organized in perceptions and preferences:

The classification is based on criteria of internal motivation (meritocratic) and structural constraints (non-meritocratic). There were two items from the battery that were excluded from the analysis as they would not fit into the classification. The first one was “having good education yourself”, since it was not clear whether this is could be due to individual motivation or system opportunities, and the second was “giving bribes”, as introduced elements of criminality that were beyond a non-meritocratic perception.

Table 1 below summarise the items classified according to the proposal based on perception and preference regarding to the meritocratic/non-meritocratic dimensions:

Table 1: Items of the ISSP meritocratic perceptions and preferences measures
Component Dimensions Item
Perception Meritocratic How important is having ambition?
How important is hard work?
Non-meritocratic How important is coming from a wealthy family?
How important is a person’s race?
How important is being born a man or a woman?
How important is knowing the right people
How important is having political connections
Preference Meritocratic How well he or she does the job?
How hard he or she works at the job?
Non-meritocratic What is needed to support a family?
Whether the person has children to support?

Methods

The estimation was performed using Confirmatory Factor Analysis (CFA). CFA was conducted using the lavaan R package (version 0.6-3; Rosseel, 2020), with diagonally weighted least squares (DWLS) estimation due to the items’ ordinal level of measurement (Kline, 2016; Rosseel, 2020). As recommended by Brown (2008), we assessed model fit by jointly considering the comparative fit index and Tucker-Lewis Index (CFI and TLI; acceptable fit > 0.95), Root of the average squared residual approximation (RMSEA; acceptable fit < 0.08), Chi-square: (p-value; acceptable fit > 0.05, and Chi-square ratio > 3).

Results

Descriptive analyses

Figure 1 shows the distribution of responses across the indicators for each perception and preference in the meritocratic and non-meritocratic dimensions. On one side, we see that there is a certain degree of importance attributed to factors such as hard work and ambition in the process of getting ahead, concentrating 94.9% and 92.8% in the fairly important to essential categories.

By contrast, for the non-meritocratic aspects, parental background showed a more dispersed opinion, where 61.2% considers that coming from a wealthy family is important, and 76% considers that having educated parents is important. On the other hand, the background dimension shows that people’ opinions are generally in the opposite sense, in which 63.7% consider that a person’s racial origin is not very important or not important at all in the process of getting ahead in life, while 35% consider it important, and this is nearly identical regarding the importance of gender. Lastly, we can observe differences in the networks dimension, in which 85.7% consider it to be important to know the right people in the process of getting ahead in life, contrasting with 50% who consider political connections to be important.

Regarding normative preferences, we observe how well a person performs his or her job and amount of effort involved concentrates 98% and 96.9% of the responses in the categories that attribute importance to these characteristics when evaluating the pay that people deserve. Finally, in relation to structural constrains, 83% attribute importance to the fact that supporting a family should be a relevant factor with respect to the pay obtained, while having children to support translates into 76.4% of importance in this dimension.

Distribution of responses in the ISSP meritocracy items

Figure 1: Distribution of responses in the ISSP meritocracy items

In terms of the association between the indicators, Figure 2 shows the polychoric correlation matrix. Firstly, we see that the correlations between the indicators are consistent with the structure of the initially proposed structure of dimensions distinguishing between the perception and preferences of meritocratic and non-meritocratic factors. In general, the associations are positive between the perception and preference dimensions, such as the importance of hard work to get ahead and the importance of how hard the work is (\(r\)=0.34), however, they tend to be weak. A further aspect to consider is the relatively low correlation between dimensions concerning meritocratic and non-meritocratic aspects, (e.g. importance of hard work with educated parents, \(r\)=0.15). However, with respect to the non-meritocratic perception indicators, we can see that the associations demonstrate dimensionality, however, it is not entirely clear. In this regard, the indicators referring to parental characteristics have a positive and stronger association (\(r\)=0.56), but this association is much weaker in comparison to the other indicators of Non-meritocratic perception (e.g. Wealthy family and Gender, \(r\)=0.01). The same holds true for the Gender and Race indicators (\(r\)=0.6), and also applies to both the indicators of knowing the right people and having political connections (\(r\)=0.55). On this point, we will return to it in the next section.

Perceptions and preferences for ISSP meritocracy items polychoric correlations

Figure 2: Perceptions and preferences for ISSP meritocracy items polychoric correlations

Confirmatory Factor Analysis

This section presents the results of the confirmatory factor analysis. First, a four-dimensional model for meritocratic and non-meritocratic perception and preferences is estimated, considering all indicators of the non-meritocratic perception dimension as a single latent variable. Then, we estimate a model that follows the proposed four latent variable measurement approach, however, the non-meritocratic perception dimension is assumed as a second-order factor for the first order latent variables parental attributes, racial and gender background and networks.

Table 2: Summary fit indices according to model
Model \(N\) Estimator \(\chi^2\) df CFI TLI RMSEA
First order 46594 DWLS 21308.535 48 0.959 0.944 0.098
Second order 46594 DWLS 4472.369 45 0.992 0.988 0.046

Table 2 shows the four-factor model fit allowing comparison of first and second-order factors. The model restricted to four first-order factor dimensions shows a regular fit (CFI=0.959, TLI=0.944, RMSEA=0.098, \(\chi^2\)(df=48)= 21308.535; more detail in Table 3) in contrast to the model considering a second-order factor for non-meritocratic perception that has a slightly better fit based on the standard fit measures (CFI=0.959, TLI=0.944, RMSEA=0.098, \(\chi^2\)(df=48)= 21308.535).

Confirmatory factor analysis of the ISSP indicators of Perceptions and Preferences for Meritocracy

Figure 3: Confirmatory factor analysis of the ISSP indicators of Perceptions and Preferences for Meritocracy

Figure 3 shows the overall results for this model, evidencing an improvement in factor loadings as well as in the overall fit of the measurement model. On one hand, we see that perceptions are correlated with preferences, but this correlation is stronger for the meritocratic dimension of perception and preference (\(r\)=0.46) than for the non-meritocratic (\(r\)=0.27), which is much stronger when looking at the correlations between meritocratic and non-meritocratic, both in perception (\(r\)=0.04) and preference (\(r\)=0.13). These results show that, on one hand, the positive correlation between perception indicates that the two are not completely opposite poles of a continuum (\(r\)=0.30), which is reflected in the correlation between preferences (\(r\)=0.25). Furthermore, it is shown that the indicators of non-meritocratic perception refer to three other sub-dimensions and cannot be completely attributed to a an entirely common factor as has been employed in previous research.

Appendix

Table 3: Factor loadings and fit measures for ISSP 2009
Factor loadings
Perception
Preference
Meritocratic Non-meritocratic Meritocratic Non-meritocratic
Importance: hard work 0.709
Importance: having ambition 0.726
Importance: wealthy family 0.694
Importance: educated parents 0.609
Importance: race 0.624
Importance: gender 0.591
Importance: knowing people 0.612
Importance: political connections 0.644
Reasons for pay: well job 0.760
Reasons for pay: hard job 0.910
Reasons for pay: support family 0.837
Reasons for pay: has children 0.918
\(\chi^2\text{(df)}\) 21308.5(48)
\(\text{CFI}\) 0.959
\(\text{TLI}\) 0.944
\(\text{RMSEA}\) 0.098
\(N\) 46594
Note: Standardized factor loadings using DWLS estimator ; CFI = Comparative fit index ;TLI = Tucker-Lewis index; RMSEA = Root mean square error of approximation